feat(gnovm): add benchmark system #1624
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #1624 +/- ##
=======================================
Coverage 56.42% 56.43%
=======================================
Files 439 439
Lines 66558 66558
=======================================
+ Hits 37558 37562 +4
+ Misses 26087 26081 -6
- Partials 2913 2915 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
|
@thehowl They won't get picked by the benchmark charts by default, you have to create a rule first. All new benchmarks will be added afterward. There is a check on the PR template pointing to the instructions on how to do it: https://github.com/gnolang/gno/blob/master/.github/pull_request_template.md?plain=1 In any case, the Github action runner needed for the tests never was deployed AFAIK. |
zivkovicmilos
approved these changes
Feb 8, 2024
3 tasks
leohhhn
pushed a commit
to leohhhn/gno
that referenced
this pull request
Feb 8, 2024
I was attempting some changes with performance ramifications and wanted to measure their impact. Eventually, they turned out to be bad performance-wise, but the code I made for measuring performance is probably still useful. This PR sets up a basic performance measurement system for the GnoVM. It leverages Go's existing benchmarking system together with templates. It allows us to measure impact on performance using standard go tools like [benchstat](https://pkg.go.dev/golang.org/x/perf/cmd/benchstat). ```console $ go test -bench . -run NONE . goos: linux goarch: amd64 pkg: github.com/gnolang/gno/gnovm/pkg/gnolang cpu: AMD Ryzen 7 PRO 4750U with Radeon Graphics BenchmarkPreprocess-16 7737 152430 ns/op BenchmarkBenchdata/fib.gno_param:4-16 47374 24664 ns/op BenchmarkBenchdata/fib.gno_param:8-16 10000 213632 ns/op BenchmarkBenchdata/fib.gno_param:16-16 100 10590462 ns/op BenchmarkBenchdata/loop.gno-16 2556032 509.3 ns/op BenchmarkBenchdata/matrix_mult.gno_param:3-16 1976 590535 ns/op BenchmarkBenchdata/matrix_mult.gno_param:4-16 740 1703166 ns/op BenchmarkBenchdata/matrix_mult.gno_param:5-16 180 6797221 ns/op BenchmarkBenchdata/matrix_mult.gno_param:6-16 36 34389320 ns/op BenchmarkCreateNewMachine-16 80518 13819 ns/op PASS ok github.com/gnolang/gno/gnovm/pkg/gnolang 16.790s ``` A few benchmarks are provided to show its functioning and benchmark some initial functions. `loop.gno` brings over the `LoopyMain` existing benchmark, and makes its output more useful by correlating it with `b.N`. Making this to have a preliminary review and open up discussions: - I removed `go_bench_data.go`, because as the name suggests it tests out things in Go and isn't actually at all related to the GnoVM. - Do we want to keep these benchmarks in `misc/` or somewhere? Or has the information gathered from these tests already been used to act, and as such they can be removed? - Was something similar already in the works? @deelawn @petar-dambovaliev - Is it OK to place the benchmarking files in `benchdata`, or would we prefer another location? - Would these benchmarks be automatically picked up for gnolang/benchmarks? cc/ @ajnavarro @albttx
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I was attempting some changes with performance ramifications and wanted to measure their impact. Eventually, they turned out to be bad performance-wise, but the code I made for measuring performance is probably still useful.
This PR sets up a basic performance measurement system for the GnoVM. It leverages Go's existing benchmarking system together with templates. It allows us to measure impact on performance using standard go tools like benchstat.
A few benchmarks are provided to show its functioning and benchmark some initial functions.
loop.gnobrings over theLoopyMainexisting benchmark, and makes its output more useful by correlating it withb.N.Making this to have a preliminary review and open up discussions:
go_bench_data.go, because as the name suggests it tests out things in Go and isn't actually at all related to the GnoVM.misc/or somewhere? Or has the information gathered from these tests already been used to act, and as such they can be removed?benchdata, or would we prefer another location?